The resolution of display devices continues to increase. For example 3840×2160 panels have become more prevalent on the market. However, almost all video sources have resolutions lower than this. The higher resolution panels typically include some form of super resolution or upscaling to convert the image/video source from low resolution to high resolution in order to match the device's resolution.
For example, super resolution techniques generate higher resolution frames from lower resolution frames. Single frame super resolution (SFSR) can recover details and preserve sharp edges with relatively low computation cost when compared with multi-frame super resolution techniques. Another example includes less complex methods of upscaling to the higher resolution. However, the upscaled image results often look less stable in the temporal domain and have more noise than the input low resolution images. These are just examples of images that may have temporal instabilities. Other types of image data may have temporal instabilities, such as decompressed image data.
The cause of the instability and the more obvious noise results from using information from a single image or an external training library instead of information from multiple frames in the temporal domain. Because of this, the above methods cannot distinguish noise from details. Therefore, if the input source has some noise, the output looks more annoying in the temporal domain during playback.
Upscaling or super resolution techniques sometimes produce artifacts because these algorithms do not have enough information to always correctly reconstruct the missing detail, resulting in a wrong or unstable operation in the processes. The artifacts change from frame-to-frame causing instability in the temporal domain. Because these techniques generally produce an enhanced image, even a slight discrepancy in texture regions among adjacent input frames will bring even more inconsistency in the temporal domains after upscaling and/or frame rate conversion.